Variations in Physician Telemedicine Provision.

This cross-sectional study assesses variation in the provision of telemedicine services among primary care physicians and quantifies the extent to which this variation may be explained by the individual physician vs temporal, patient, or visit factors.

Pilot assessment of an on-demand telehealth 'left without being seen' follow-up programme.

INTRODUCTION: On-demand telehealth can have a high rate of patients requesting visits and dropping off without being seen by a provider, especially during the COVID-19 pandemic. METHODS: On-demand telehealth requests made to a large healthcare system in the USA between 15 March 2020 and 31 May 2020 were included for analysis with a focus on patients who were defined as left without being seen (LWBS). As part of a pilot program a registered nurse attempted to call LWBS patients within 24 hours of their telehealth request and asked if they were ok, if they sought care for their original visit reason, what that care was, or if they still needed guidance. This information and patient demographics were analyzed. RESULTS: During the study period there were 21,610 completed on-demand telehealth visits and 1852 patients for whom there were LWBS attempted follow-ups. Most patients LWBS for a reason that originated from the patient and not associated with the provider or telehealth platform. The mean wait time for LWBS patients was 12.4 min compared to patients waiting 15.1 min before engaging with a provider to complete a visit. Of the 1852 total LWBS patients in the follow-up programme, 819 (44.2%) were successfully contacted with a follow-up phone call. Most of these patients (63.2%) already completed or planned to complete a telehealth visit, 13.6% indicated they no longer needed to see a provider, and 12.8% planned or already completed an in-person visit. Only 2.2% went to an emergency department. DISCUSSION: Results suggest patients can effectively self-manage their care needs.

Identifying Electronic Medication Administration Record (eMAR) Usability Issues from Patient Safety Event Reports.

BACKGROUND: Improving our understanding of the association between medication errors and health information technology (health IT) usability has the potential to reduce errors and improve patient safety. This study used patient safety event reports (PSEs) to investigate the contribution of usability challenges associated with the electronic medication administration record (eMAR) to medication errors. METHODS: Free-text descriptions of 849 medication-related PSEs selected from 2.3 million reports were analyzed. Coders identified the specific health IT components, usability challenge categories, and nuanced usability themes that contributed to each PSE. Thematic analysis was conducted to refine categorizations and identify emerging themes. Final analysis was limited to PSEs involving a contribution from eMAR, either as the point of origin or as a downstream contributor to error. RESULTS: eMAR contributed to 473 PSEs. eMAR was the point of origin for 84 (17.8% of 473) PSEs. Usability challenge categories included Workflow support (n = 52, 11.0%) and Display/Visual clutter (n = 30, 6.3%). eMAR contributed downstream from the point of origin in 389 (82.2% of 473) PSEs, with errors stemming primarily from Pharmacy IT and computerized provider order entry (CPOE). Prominent secondary eMAR-associated usability challenges included Display/Visual clutter (n = 327, 69.1%) and Alerting (n = 32, 6.8%). CONCLUSION: This study identified several eMAR usability challenges, through the analysis of PSEs, that contribute to medication errors. Findings highlight the critical need for improving the eMAR user interface. Improved interface design, better vendor usability testing, eMAR-focused certification testing, consideration of work system factors, and eMAR-focused usability and safety testing by health care facilities can improve eMAR technology and patient safety.

Identifying Health Information Technology Usability Issues Contributing to Medication Errors Across Medication Process Stages.

OBJECTIVE: Different health information technology (health IT) systems are intended to support medication ordering, reviewing, and administration. We sought to identify the types of medication errors associated with health IT use, whether they reached the patient, where in the medication process those errors occurred, and the specific usability issues contributing to those errors. METHODS: Patient safety event reports from more than 595 healthcare facilities entered between January 2013 and September 2018 were analyzed. We computationally identified reports associated with health IT intended to support the medication process, including computerized provider order entry, electronic medication administration record, and barcode medication administration. From these, 2700 reports were manually reviewed to determine the type of medication error, medication process stage, and health IT usability issue. RESULTS: Of the 2700 manually reviewed reports, 1508 (55.9%) described a medication error that was associated with health IT use and 750 (49.7%) reached the patient. Improper dose errors were frequent (1214 of 1508, 80.5%) with most errors during ordering (673 of 1508, 44.6%) and reviewing medications (639 of 1508, 42.4%). Most health IT-associated medication error reports described usability issues (n = 1468 of 1508, 97.3%) including data entry, workflow support, and alerting. Data entry usability issues impacted few medication process stages, whereas workflow support and alerting impacted several stages. CONCLUSIONS: Health IT usability issues are a prevalent contributing factor to medication errors, many of which reach the patient. Data entry, workflow support, and alerting should be prioritized during usability and safety optimization efforts.

Understanding the Types and Effects of Clinical Interruptions and Distractions Recorded in a Multihospital Patient Safety Reporting System.

OBJECTIVES: Interruptions and distractions have been shown to be a frequent occurrence across health care and have been linked to negative outcomes that create potential patient safety risks. Although observational studies have catalogued interruption frequency and source, the impact of an interruption is difficult to observe. We analyzed patient safety event (PSE) reports related to interruptions to identify clinical processes reported to be frequently interrupted and the reported outcomes of those interruptions. METHODS: We retrospectively analyzed PSE reports entered by frontline staff between January 2013 and January 2016. Of 79,428 total PSEs entered, 220 reports were identified using keyword matching and subsequent manual review as being directly related to a clinical interruption. Categories were developed to identify the cause of the interruption, task being interrupted, and the result of the interruption. Percentages were calculated. RESULTS: Nurses were most often reported to be interrupted in the PSEs (50%). General distractions (43.2%) or high workload (18.6%) were most commonly noted to interrupt the individual's work. The interrupted activity was most often a medication task (50.9%), frequently in the administration phase (24.1%), or the ordering phase (16.8%). The most common medication error was wrong dose administration (14.4% of total medication-related errors). Laboratory processes were reported to be disturbed by interruptions in 22.7% of reports, and this frequently resulted in mislabeling of specimens (75% of laboratory-related errors). CONCLUSIONS: This retrospective review of PSE reports involving interruptions of clinical activities reveals that interruptions affect a variety of aspects of patient care and can help to guide future work on interruption management.

Workflow Analysis Driven Recommendations for Integration of Electronically-Enhanced Sexually Transmitted Infection Screening Tools in Pediatric Emergency Departments.

Adolescents are disproportionately affected by sexually transmitted infections (STIs). Failure to diagnose and treat STIs in a timely manner may result in serious sequelae. Adolescents frequently access the emergency department (ED) for care. Although ED-based STI screening is acceptable to both patients and clinicians, understanding how best to implement STI screening processes into the ED clinical workflow without compromising patient safety or efficiency is critical. The objective of this study was to conduct direct observations documenting current workflow processes and tasks during patient visits at six Pediatric Emergency Care Applied Research Network (PECARN) EDs for site-specific integration of STI electronically-enhanced screening processes. Workflow observations were captured via TaskTracker, a time and motion electronic data collection application that allows researchers to categorize general work processes and record multitasking by providing a timestamp of when tasks began and ended. Workflow was captured during 118 patient visits across six PECARN EDs. The average time to initial assessment by the most senior provider was 76 min (range 59-106 min, SD = 43 min). Care teams were consistent across sites, and included attending physicians, advanced practice providers, nurses, registration clerks, technicians, and students. A timeline belt comparison was performed. Across most sites, the most promising implementation of a STI screening tool was in the patient examination room following the initial patient assessment by the nurse.

Preliminary Assessment of a Telehealth Approach to Evaluating, Treating, and Discharging Low-Acuity Patients With Suspected COVID-19.

BACKGROUND: Telemedicine is uniquely positioned to address challenges posed to emergency departments (EDs) by the Coronavirus Disease 2019 (COVID-19) pandemic. By reducing in-person contact, it should decrease provider risk of infection and preserve personal protective equipment (PPE). OBJECTIVES: To describe and assess the early results of a novel telehealth workflow in which remote providers collaborate with in-person nursing to evaluate and discharge well-appearing, low-risk ED patients with suspected COVID-19 infection. METHODS: Retrospective chart review was completed 3 weeks after implementation. Metrics include the number of patients evaluated, number of patients discharged without in-person contact, telehealth wait time and duration, collection of testing, ED length of stay (ED-LOS), 72-h return, number of in-person health care provider contacts, and associated PPE use. RESULTS: Among 302 patients evaluated by telehealth, 153 patients were evaluated and discharged by a telehealth provider with reductions in ED-LOS, PPE use, and close contact with health care personnel. These patients had a 62.5% shorter ED-LOS compared with other Emergency Severity Index level 4 patients seen over the same time period. Telehealth use for these 153 patients saved 413 sets of PPE. We observed a 3.9% 72-h revisit rate. One patient discharged after telehealth evaluation was hospitalized on a return visit 9 days later. CONCLUSION: Telehealth can be safely and efficiently used to evaluate, treat, test, and discharge ED patients suspected to have COVID-19. This workflow reduces infection risks to health care providers, PPE use, and ED-LOS. Additionally, it allows quarantined but otherwise well clinicians to continue working.

Rapid development of visualization dashboards to enhance situation awareness of COVID-19 telehealth initiatives at a multihospital healthcare system.

The COVID-19 pandemic has led to the rapid expansion of telehealth services as healthcare organizations aim to mitigate community transmission while providing safe patient care. As technology adoption rapidly increases, operational telehealth teams must maintain awareness of critical information, such as patient volumes and wait times, patient and provider experience, and telehealth platform performance. Using a model of situation awareness as a conceptual foundation and a user-centered design approach we describe our process for rapidly developing and disseminating dashboard visualizations to support telehealth operations. We used a 5-step process to gain domain knowledge, identify user needs, identify data sources, design and develop visualizations, and iteratively refine these visualizations. Through this process we identified 3 distinct stakeholder groups and designed and developed visualization dashboards to meet their needs. Feedback from users demonstrated the dashboard's support situation awareness and informed important operational decisions. Lessons learned are shared to provide other organizations with insights from our process.

Exploration and Initial Development of Text Classification Models to Identify Health Information Technology Usability-Related Patient Safety Event Reports.

BACKGROUND: With the pervasive use of health information technology (HIT) there has been increased concern over the usability and safety of this technology. Identifying HIT usability and safety hazards, mitigating those hazards to prevent patient harm, and using this knowledge to improve future HIT systems are critical to advancing health care. PURPOSE: The purpose of this work is to demonstrate the feasibility of a modeling approach to identify HIT usability-related patient safety events (PSEs) from the free-text of safety reports and the utility of such models for supporting patient safety analysts in their analysis of event data. METHODS: We evaluated three feature representations (bag-of-words [BOWs], topic modeling, and document embeddings) to classify HIT usability-related PSE reports using 5,911 manually annotated reports. Model results were reviewed with patient safety analysts to gather feedback on their usefulness and integration into workflow. RESULTS: The combination of term frequency-inverse document frequency BOWs and document embedding features modeled with support vector machine (SVM) with radial basis function (RBF) had the highest overall precision-recall area under the curve (AUC) and f1-score, 72 and 66%, respectively. Using only document embedding features achieved a similar precision-recall AUC and f1-score performance with the SVM RBF model, 70 and 66%, respectively. Models generally favored specificity and sensitivity over precision. Patient safety analysts found the model results to be useful and offered three suggestions on how it can be integrated into their workflow at the point of report entry, in a visual dashboard layer, and to support data retrievals. CONCLUSION: Text mining and document embeddings can support identification of HIT usability-related PSE reports. The positive feedback received on the HIT usability model shows its potential utility in real-world applications.

The Patient in Precision Medicine: A Systematic Review Examining Evaluations of Patient-Facing Materials.

Precision medicine (PM) has the potential to tailor healthcare to the individual patient by using their genetic information to guide treatment choices. However, this process is complex and difficult to understand for patients and providers alike. With a recent push in the healthcare community to understand the patient experience and engage patients in their care, it is important to give patients the opportunity to learn about PM. We performed a systematic review to identify previous work assessing the quality of patient-facing PM materials from 2008 to July 2018. Ten studies were identified, which used varying methods and measures. A qualitative assessment was conducted to compare key elements of the studies, including study design, characteristics of the participant population, what measurements were used to assess the PM materials, understandability, preference, psychological reactions, and the type of PM materials being assessed. The studies identified provide important groundwork by highlighting consistent aspects of design that aid in comprehension. Eight of the ten studies focused on the content and organization of genomic test results, while the remaining two assessed educational tools. Two main design elements that appeared across the studies were appropriately designed visual aids and simplified language. The studies identified were limited by the participant populations that were used, which were primarily white and well educated. Only one study attempted to oversample patient populations typically underrepresented in this type of research. Through our systematic review, it is evident that the breadth of knowledge in this field is limited in scope and that more work must be done to ensure that patients can engage in their care when faced with PM.

Identifying health information technology related safety event reports from patient safety event report databases.

OBJECTIVE: The objective of this paper was to identify health information technology (HIT) related events from patient safety event (PSE) report free-text descriptions. A difference-based scoring approach was used to prioritize and select model features. A feature-constraint model was developed and evaluated to support the analysis of PSE reports. METHODS: 5287 PSE reports manually coded as likely or unlikely related to HIT were used to train unigram, bigram, and combined unigram-bigram logistic regression and support vector machine models using five-fold cross validation. A difference-based scoring approach was used to prioritize and select unigram and bigram features by their relative importance to likely and unlikely HIT reports. A held-out set of 2000 manually coded reports were used for testing. RESULTS: Unigram models tended to perform better than bigram and combined models. A 300-unigram logistic regression had comparable classification performance to a 4030-unigram SVM model but with a faster relative run-time. The 300-unigram logistic regression model evaluated with the testing data had an AUC of 0.931 and a F1-score of 0.765. DISCUSSION: A difference-based scoring, prioritization, and feature selection approach can be used to generate simplified models with high performance. A feature-constraint model may be more easily shared across healthcare organizations seeking to analyze their respective datasets and customized for local variations in PSE reporting practices. CONCLUSION: The feature-constraint model provides a method to identify HIT-related patient safety hazards using a method that is applicable across healthcare systems with variability in their PSE report structures.

Using Active Learning to Identify Health Information Technology Related Patient Safety Events.

The widespread adoption of health information technology (HIT) has led to new patient safety hazards that are often difficult to identify. Patient safety event reports, which are self-reported descriptions of safety hazards, provide one view of potential HIT-related safety events. However, identifying HIT-related reports can be challenging as they are often categorized under other more predominate clinical categories. This challenge of identifying HIT-related reports is exacerbated by the increasing number and complexity of reports which pose challenges to human annotators that must manually review reports. In this paper, we apply active learning techniques to support classification of patient safety event reports as HIT-related. We evaluated different strategies and demonstrated a 30% increase in average precision of a confirmatory sampling strategy over a baseline no active learning approach after 10 learning iterations.

An Analysis of Patient Safety Incident Reports Associated with Electronic Health Record Interoperability

Background: With the widespread use of electronic health records (EHRs) for many clinical tasks, interoperability with other health information technology (health IT) is critical for the effective delivery of care. While it is generally recognized that poor interoperability negatively impacts patient care, little is known about the specific patient safety implications. Understanding the patient safety implications will help prioritize interoperability efforts around architectures and standards. Objectives: Our objectives were to (1) identify patient safety incident reports that reflect EHR interoperability challenges with other health IT, and (2) perform a detailed analysis of these reports to understand the health IT systems involved, the clinical care processes impacted, whether the incident occurred within or between provider organizations, and the reported severity of the patient safety events. Methods: From a database of 1.735 million patient safety event (PSE) reports spanning multiple provider organizations, 2625 reports that were indicated as being health IT related by the event reporter were reviewed to identify EHR interoperability related reports. Through a rigorous coding process 209 EHR interoperability related events were identified and coded. Results: The majority of EHR interoperability PSE reports involved interfacing with pharmacy systems (i.e. medication related), followed by laboratory, and radiology. Most of the interoperability challenges in these clinical areas were associated with the EHR receiving information from other health IT systems as opposed to the EHR sending information to other systems. The majority of EHR interoperability challenges were within a provider organization and while many of the safety events reached the patient, only a few resulted in patient harm. Conclusions: Interoperability efforts should prioritize systems in pharmacy, laboratory, and radiology. Providers should recognize the need to improve EHRs interfacing with other health IT systems within their own organization.